[PEP 695] Fix incorrect Variance Computation with Polymorphic Methods. #19466

randolf-scholz · 2025-07-16T14:46:40Z

My idea for fixing it was to replace typ = find_member(member, self_type, self_type) with typ = find_member(member, self_type, plain_self) inside the function infer_variance, where plain_self is the type of self without any type variables.

To be frank, I do not myself 100% understand why it works / if it is safe, but below is my best effort explanation.
Maybe a better solution is to substitute all function variables with UninhabitedType()?
But I am not sure how to do this directly, since the type is only obtained within find_member.

According to the docstring of find_member_simple:

Find the member type after applying type arguments from 'itype', and binding 'self' to 'subtype'. Return None if member was not found.

Since plain_self is always a supertype of the self type, however it may be parametrized, the typ we get this way should be compatible with the typ we get using the concrete self_type. However, by binding self only to plain_self, it replaces substituted polymorphic variables with Never.

Examples:

class Foo[T]:
    def new[S](self: "Foo[S]", arg: list[S]) -> "Foo[S]": ...

class Bar[T]:
    def new(self, arg: list[T]) -> "Foo[T]": ...

With this patch:

Foo.new becomes
- def [S] (self: tmp_d.Foo[Never], arg: builtins.list[Never]) -> tmp_d.Foo[Never] in typeops.py#L470
- def (arg: builtins.list[Never]) -> tmp_d.Foo[Never] in subtypes.py#L2211
Bar.new becomes def (arg: builtins.list[T`1]) -> tmp_d.Bar[T`1] (✅)

Without this patch:

Foo.new becomes
- def [S] (self: tmp_d.Foo[T`1], arg: builtins.list[T`1]) -> tmp_d.Foo[T`1] in typeops.py#L470 (❌)
- def (arg: builtins.list[T`1]) -> tmp_d.Foo[T`1] in subtypes.py#L2211 (❌)
Bar.new becomes def (arg: builtins.list[T`1]) -> tmp_d.Bar[T`1] (✅)

Another way to think about it is we can generally assume a signature of the form:

class Class[T]:
    def method[S](self: Class[TypeForm[S, T]], arg: TypeForm[S, T]) -> TypeForm[S, T]: ...

Now, given self_type is Class[T], it first solves Class[T] = Class[TypeForm[S, T]] for S inside bind_self, giving us some solution S(T), and then substitutes it giving us some non-polymorphic method

def method(self: Class[T], arg: TypeForm[T]) -> TypeForm[T]

and then drops the first argument, so we get the bound method method(arg: TypeForm[T]) -> TypeForm[T].

By providing the plain_self, the solution we get is S = Never, which solve the problem.

mypy/subtypes.py

Co-authored-by: Stanislav Terliakov <[email protected]>

sterliakov · 2025-09-02T00:02:33Z

Also fixes #18334 and maybe something else.

randolf-scholz · 2025-09-16T12:50:35Z

@sterliakov I added #18334 as a unit test.

github-actions · 2025-10-06T15:53:32Z

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

randolf-scholz · 2025-10-06T16:08:31Z

@sterliakov This and a few other small PRs (#19517, #19471, #19449) have been sitting since mid-July, I'm guessing you are rather swamped. Who else should I ask for review?

sterliakov · 2025-10-06T22:22:48Z

I am a bit swamped indeed, and also I don't have merge powers here anyway:)

This change is really non-trivial. I have already looked at this PR and, tbh, I'm still not 100% certain that binding self to Any-filled self is the correct move here. It makes some sense to me, but I'd love to see more tests with manually verified logic.

I just took another look, and IMO the following snippet will be handled incorrectly:

class Mixed[T, U]:
    def new[S](self: "Mixed[S, U]", key: S, val: U) -> None: ...

x = Mixed[str, int]()
x.new(object(), 0)  # Should error
# So this upcast is not really an upcast and should be rejected, T should be contravariant?
y: Mixed[object, int] = x
y.new(object(), 0)  # Should not error

check_contra: Mixed[str, int] = Mixed[object, int]()
check_co: Mixed[object, int] = Mixed[str, int]()

It's not strictly incorrect (because resolving S to object would be valid for x.new() call), but is handled by mypy that way, and errors should never disappear when a value is upcasted (assigned to without ignore, type error or cast) to some wider type.

I didn't try to compare this to Pyright or other type checkers.

I'd suggest to also ping @JukkaL for review - he authored a huge part of PEP695 implementation, including this inference.

randolf-scholz · 2025-10-07T08:25:31Z

I tested a variation of your example pyright-playground, mypy-playground

from typing import cast

class Mixed[T, U]:
    def new[S](self: "Mixed[S, U]", key: S, val: U) -> None: ...

def test_co(x: Mixed[str, int]) -> Mixed[object, int]:
    return x                    # master: ❌ PR: ✅, pyright: ✅

def test_contra(x: Mixed[object, int]) -> Mixed[str, int]:
    return x                    # master: ✅ PR: ❌, pyright: ❌

def test_now_sub(y: Mixed[object, int]) -> None:
    # str value is assignable to object type.
    y.new(key=str(), val=0)     # master: ✅ PR: ✅, pyright: ✅

def test_new_super(x: Mixed[str, int]) -> None:
    # object type is not assignable to str variable.
    x.new(key=object(), val=1)  # master: ❌ PR: ❌, pyright: ❌
    
def test_new_upcast(x: Mixed[str, int]) -> None:
    # technically, one could upcast first:
    z: Mixed[object, int] = x   # master: ❌ PR: ✅, pyright: ✅
    z = Mixed[object, int]()
    z.new(key=object(), val=1)  # master: ✅ PR: ✅, pyright: ✅

So both pyright and the PR treat T as covariant, whereas on master mypy treats it as contravariant. The new-cast still fails because object cannot be assigned to key. On PR and pyright, we can make it work by upcasting first, which is how it should be, since calling a method isn't allowed to mutate the type.

use plain_self as subtitution type for self

d7b88cc

sterliakov reviewed Jul 16, 2025

View reviewed changes

mypy/subtypes.py Outdated Show resolved Hide resolved